Multi-track hinting for receiver-driven streaming system

ABSTRACT

A receiver-driven streaming method is provided. The method includes receiving an original coded video stream from a transmitter  102  at a receiver  104.  An available bit rate is measured at the receiver  104.  A request for a different coded video stream and a switching stream is sent from the receiver  104  to the transmitter  102  based on the available bit rate. The requested switching stream is received from the transmitter  102  at the receiver  104.  The requested coded video stream is received from the transmitter  102  at the receiver  104.    
     According to one embodiment, switching streams  132   a - 132   f  are stored separately from coded video streams  130   a - 130   c,  and each switching stream  132   a - 132   f  and coded video stream  130   a - 130   c  is stored in a separate track from each other. In this way, switching streams  132   a - 132   f  maybe generated offline. Also, the receiver  104  controls the streams provided by the transmitter  102  by requesting specific switching streams  132   a - 132   f  and coded video streams  130   a - 103   c.

TECHNICAL FIELD OF THE INVENTION

The present invention relates generally to multi-track hinting andstreaming systems and, more particularly, to a multi-track hinting forreceiver-driven streaming system for non-scalable video coding usingswitching frames and method.

BACKGROUND OF THE INVENTION

With the rapid development of broadband Internet technologies, videostreaming is envisioned to become the dominant Internet application inthe near future. Similarly, the falling cost of wireless local areanetwork (WLAN) products has led to their increased use in consumerhomes. Also, although currently most WLANs are predominantly used fordata transfer, the higher bandwidth provided by new WLAN technologies,such as IEEE 802.11a and IEEE 802.11g, may ultimately lead to theirincreasing use for video transmission. Furthermore, future wirelessvideo applications will have to work over an open, layered,Internet-style network with a wired backbone and wireless extensions.Therefore, common protocols will have to be used for transmission acrossboth the wired and wireless portions of the network. These protocolswill most likely be future extensions of the existing protocols that arebased on the Internet Protocol (IP).

Consequently, due to the inherent resource sharing nature of theInternet and wireless networks, multimedia communications of the futurewill mainly use variable bandwidth channels. Hence, if streaming ofvideo content is performed over this type of network, the instantaneousdata rate must frequently be tailored to fit the available resources.This can be achieved in a very flexible way by the approach of scalablecoding. Scalable video-coding schemes are able to provide a simple andflexible framework for transmission over heterogeneous networks.

However, in order to provide the required adaptation to bandwidthvariations, device characteristics and user requirements, multiplenon-scalable video coding with a different data rate needs to betransmitted using an appropriate streaming architecture. The MPEG-4Systems Group has developed and standardized the streaming strategy fornon-scalable coded video over IP networks. However, this strategy isunable to efficiently adapt to channel conditions, complexityconstraints and user preferences. In addition, conventional systems suchas these require the transmitter to be responsible for inserting thecorrect switching frames into the video stream to accomplish smoothswitching.

SUMMARY OF THE INVENTION

In accordance with the present invention, a multi-track hinting forreceiver-driven streaming system for non-scalable video coding usingswitching frames and method are provided that substantially eliminate orreduce disadvantages and problems associated with conventional systemsand methods. In particular, each receiver may independently adapt itsreceived video optimally to its own perceived network conditions.

According to one embodiment of the present invention, a receiver-drivenstreaming method is provided. The method includes receiving an originalcoded video stream from a transmitter at a receiver. An available bitrate is measured at the receiver. A request for a different coded videostream and a switching stream is sent from the receiver to thetransmitter based on the available bit rate. The requested switchingstream is received from the transmitter at the receiver. The requestedcoded video stream is received from the transmitter at the receiver.

According to another embodiment of the present invention, areceiver-driven streaming method is provided that includes transmittingan original coded video stream from a transmitter to a receiver. Arequest for a different coded video stream and a switching stream isreceived from the receiver at the transmitter. The requested switchingstream is transmitted from the transmitter to the receiver. Therequested coded video stream is transmitted from the transmitter to thereceiver.

Technical advantages of one or more embodiments of the present inventioninclude providing an improved streaming system. In a particularembodiment, coded video streams at different bit rates and switchingstreams are stored in separate tracks. In this way, a receiver-drivenstreaming system is provided that may respond more quickly to networkchanges than a transmitter-driven streaming system, thereby achievingbetter performance. This is due to the ability of each receiver toindependently adapt its received video optimally to its own perceivednetwork conditions.

Other technical advantages will be readily apparent to one skilled inthe art from the following figures, description, and claims.

Before undertaking the DETAILED DESCRIPTION, it may be advantageous toset forth definitions of certain words and phrases used throughout thispatent document: the terms “include” and “comprise,” as well asderivatives thereof, mean inclusion without limitation; the term “or,”is inclusive, meaning and/or; the phrases “associated with” and“associated therewith,” as well as derivatives thereof, may mean toinclude, be included within, interconnect with, contain, be containedwithin, connect to or with, couple to or with, be communicable with,cooperate with, interleave, juxtapose, be proximate to, be bound to orwith, have, have a property of, or the like; and the term “controller”means any device, system or part thereof that controls at least oneoperation, such a device may be implemented in hardware, firmware orsoftware, or some combination of at least two of the same. It should benoted that the functionality associated with any particular controllermay be centralized or distributed, whether locally or remotely. Inparticular, a controller may comprise one or more data processors, andassociated input/output devices and memory, that execute one or moreapplication programs and/or an operating system program. Definitions forcertain words and phrases are provided throughout this patent document,those of ordinary skill in the art should understand that in many, ifnot most instances, such definitions apply to prior, as well as futureuses of such defined words and phrases.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of the present invention and itsadvantages, reference is now made to the following description taken inconjunction with the accompanying drawings, wherein like referencenumerals represent like parts, in which:

FIG. 1 is a block diagram illustrating a receiver-driven streamingsystem in accordance with one embodiment of the present invention;

FIGS. 2A-B are block diagrams illustrating a multi-track hinting fileformat in accordance with one embodiment of the present invention;

FIG. 3 is a flow diagram illustrating a receiver-driven streaming methodfrom the perspective of the transmitter of FIG. 1 in accordance with oneembodiment of the present invention; and

FIG. 4 is a flow diagram illustrating a receiver-driven streaming methodfrom the perspective of one of the receivers of FIG. 1 in accordancewith one embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

FIGS. 1 through 4, discussed below, and the various embodiments used todescribe the principles of the present invention in this patent documentare by way of illustration only and should not be construed in any wayto limit the scope of the invention. Those skilled in the art willunderstand that the principles of the present invention may beimplemented in any suitably arranged streaming system.

FIG. 1 is a block diagram illustrating a receiver-driven streamingsystem 100 in accordance with one embodiment of the present invention.The system 100 comprises a transmitter 102, at least one receiver 104and a network 106 for providing communication between the transmitter102 and the receivers 104. It will be understood that the system 100 maycomprise any other suitable components of a streaming system, such asother suitable servers coupled to the network 106, without departingfrom the scope of the present invention.

The transmitter 102 comprises a processor 120 and a memory 122. Theprocessor 120 is operable to code video streams at different bit rates,store the coded video streams in separate tracks in the memory 122,generate switching streams for the coded video streams, and store theswitching streams in separate tracks in the memory 122. The transmitter102 is also operable to generate and store a hinting track (notillustrated in FIG. 1) for each coded video stream and switching stream.The processor 120 is also operable to transmit the coded video streamsand the switching streams to the receivers 104 through the network 106.

The memory 122 may comprise random access memory (RAM), a combination ofRAM and read only memory (ROM), or any other suitable form of memory.According to one embodiment, the memory 122 may comprise a non-volatileRAM, such as flash memory. The memory 122 is operable to store aplurality of coded video streams 130 and a plurality of switchingstreams 132.

For the illustrated embodiment, the memory 122 stores three coded videostreams 130: one coded video stream 130 a at a first bit rate (CVS-BR1),one coded video stream 130 b at a second bit rate (CVS-BR2), and onecoded video stream 130 c at a third bit rate (CVS-BR3). In addition, theillustrated memory 122 stores six switching streams 132, one for each ofthe possible bit rate changes between the three bit rates for the threecoded video streams 130. However, it will be understood that the memory122 may store any suitable number of coded video streams 130 andswitching streams 132 without departing from the scope of the presentinvention.

For example, even for an embodiment in which the memory 122 stores threecoded video streams 130, the memory 122 may store only four switchingstreams 132 a,c,d,f. For this embodiment, coded video streams 130 mayonly be changed to an adjacent coded video stream 130, instead ofallowing the coded video stream 130 a to be changed directly to thecoded video stream 130 c, for example. Thus, the coded video stream 130a would first be changed to the coded video stream 130 b and thenchanged to the coded video stream 130 c.

Each receiver 104 comprises a processor 140 that is operable to receivea coded video stream 130 from the transmitter 102 through the network106, to measure the available bit rate for the receiver 104 from thenetwork 106, and to request a switching stream 132 and a different codedvideo stream 130 from the transmitter 102 based on the available bitrate. Thus, each receiver 104 may independently optimize its videostream being provided by the transmitter 102 based on the networkconditions for that receiver 104. As used herein, “each” means every oneof at least a subset of the identified items.

In one embodiment, the network 106 comprises a packet data network, suchas the Internet, or other suitable network. However, the network 106 mayalso comprise any interconnection found on any computer network such asa local area network (LAN), a wide area network (WAN), or any othercommunications and data exchange systems created by connecting two ormore computers.

The transmitter 102 and the receivers 104 are operable to communicatewith the network 106 over communication lines 108, which may be any typeof communication link capable of supporting data transfer. In oneembodiment, the communication lines 108 may comprise, alone or incombination, Integrated Services Digital Network (ISDN), AsymmetricDigital Subscriber Line (ADSL), T1 or T3 communication lines, hardwirelines, or telephone links. It will be understood that the communicationlines 108 may comprise other suitable types of data communication links.The communication lines 108 may also connect to a plurality ofintermediate servers between the network 106 and the transmitter 102 andthe receivers 104.

At least a portion of the transmitter 102 and/or the receiver 104 maycomprise logic encoded in media. The logic comprises functionalinstructions for carrying out program tasks. The media comprisescomputer disks or other computer-readable media, application-specificintegrated circuits, field-programmable gate arrays, digital signalprocessors, other suitable specific or general purpose processors,transmission media or other suitable media in which logic may be encodedand utilized.

FIGS. 2A-B are block diagrams illustrating a multi-track hinting fileformat 142 in accordance with one embodiment of the present invention.For description purposes, FIG. 2A illustrates the file format 142 withrespect to the coded video streams 130, while FIG. 2B illustrates thefile format 142 with respect to the switching streams 132. However, itwill be understood that the file format 142 comprises the combination ofthe two illustrations.

The illustrated embodiment of the file format 142 corresponds to theembodiment in which the transmitter 102 stores three coded video streams130 and six switching streams 132. Thus, it will be understood that thefile format 142 may be altered so as to accommodate different numbers ofcoded video streams 130 and/or switching streams 132.

In addition to the three coded video streams 130, the file format 142 ofFIG. 2A comprises three hinting tracks 160. As indicated by the arrows,each hinting track 160 corresponds to one of the coded video streams130.

Similarly, in addition to the six switching streams 132, the file format142 of FIG. 2B comprises six hinting tracks 162. As indicated by thearrows, each hinting track 162 corresponds to one of the switchingstreams 132. Thus, the combined file format 142 provides a separatehinting track 160, 162 for each of these streams 130, 132.

FIG. 3 is a flow diagram illustrating a receiver-driven streaming methodfrom the perspective of the transmitter 102 in accordance with oneembodiment of the present invention. The method begins at step 200 wherethe transmitter 102 codes a video stream for transmission to thereceiver 104. At step 202, the transmitter 102 stores the coded videostream 130 in the memory 122 and also stores the hinting track for thecoded video stream 130.

At decisional step 204, a determination is made regarding whether or notall coded video streams 130 at all desired bit rates have been stored inthe memory 122. If the video stream has not been coded and stored at allthe desired bit rates, the method follows the No branch from decisionalstep 204 and returns to step 200 where the transmitter 102 codes thevideo stream at another bit rate and then to step 202 where thetransmitter 102 stores the coded video stream 130 in a separate trackfrom any other coded video stream 130 and also stores the hinting trackfor the coded video stream 130.

However, if the video stream has been coded and stored in separatetracks at all the desired bit rates, the method follows the Yes branchfrom decisional step 204 to step 206. At step 206, the transmitter 102stores a switching stream 132 for switching between coded video streams130 in the memory 122 and also stores the hinting track for theswitching stream 132.

At decisional step 208, a determination is made regarding whether or notall desired switching streams 132 have been stored in the memory 122. Ifall the desired switching streams have not been stored, the methodfollows the No branch from decisional step 208 and returns to step 206where the transmitter 102 stores another switching stream 132 forswitching between additional coded video streams 130 in a separate trackfrom any other switching stream 132 in the memory 122 and also storesthe hinting track for the switching stream 132.

However, if all the desired switching streams have been stored inseparate tracks, the method follows the Yes branch from decisional step208 to step 210. At step 210, the transmitter 102 transmits an originalcoded video stream 130 to the receiver 104. In addition, the transmitter102 transmits to the receiver 104 data to inform the receiver 104 of thedifferent bit rates corresponding to the stored coded video streams 130.

At decisional step 212, a determination is made regarding whether or nota request for a different coded video stream 130 has been received fromthe receiver 104. In addition to the different coded video stream 130,the request comprises a request for the switching stream 132 thatcorresponds to the difference between the original coded video stream130 and the requested coded video stream 130. If no such request isreceived, the method follows the No branch from decisional step 212 andcomes to an end. However, if such a request is received, the methodfollows the Yes branch from decisional step 212 to step 214.

At step 216, the transmitter 102 transmits the requested switchingstream 132 to the receiver 104. At step 218, the transmitter 102transmits the requested coded video stream 130 to the receiver 104, atwhich point the method returns to decisional step 212 to determinewhether or not an additional request for another coded video stream 130has been received.

FIG. 4 is a flow diagram illustrating a receiver-driven streaming methodfrom the perspective of the receiver 104 in accordance with oneembodiment of the present invention. The method begins at step 300 wherethe receiver 104 receives the original coded video stream 130 from thetransmitter 102. In addition, the receiver 104 receives data thatinforms the receiver 104 of the different bit rates corresponding to thethe coded video streams 130 stored in the transmitter 104. At step 302,the receiver 104 measures the available bit rate for the receiver 104from the network 106.

At decisional step 304, a determination is made regarding whether or notthe receiver 104 should request a switch in bit rates for the codedvideo stream 130 based on the available bit rate. If the receiver 104decides not to request a switch in bit rates, the method follows the Nobranch from decisional step 304 and returns to step 302 where theavailable bit rate continues to be measured. However, if the receiver104 decides to request a switch in bit rates, the method follows the Yesbranch from decisional step 304 to step 306.

At step 306, the receiver 104 sends a request for a different codedvideo stream 130 to the transmitter 102, with the requested coded videostream 130 based on the available bit rate. In addition to the differentcoded video stream 130, the request comprises a request for theswitching stream 132 that corresponds to the difference between theoriginal coded video stream 130 and the requested coded video stream130. At step 308, the receiver 104 receives the requested switchingstream 132 from the transmitter 102. At step 310, the receiver 104receives the requested coded video stream 130, after which the methodreturns to step 302 where the available bit rate continues to bemeasured.

Although the present invention has been described with severalembodiments, various changes and modifications may be suggested to oneskilled in the art. It is intended that the present invention encompasssuch changes and modifications as fall within the scope of the appendedclaims.

1. A receiver-driven streaming method, comprising: receiving an originalcoded video stream from a transmitter at a receiver; measuring anavailable bit rate at the receiver; sending a request for a differentcoded video stream and a switching stream based on the available bitrate from the receiver to the transmitter; receiving the requestedswitching stream from the transmitter at the receiver; and receiving therequested coded video stream from the transmitter at the receiver. 2.The method of claim 1, the receiver and the transmitter coupled to eachother through a network.
 3. The method of claim 1, the transmitteroperable to store at least three coded video streams and at least fourswitching streams for each of a plurality of video streams.
 4. Themethod of claim 1, the transmitter operable to store at least threecoded video streams and at least six switching streams for each of aplurality of video streams.
 5. A receiver-driven streaming method,comprising: transmitting an original coded video stream from atransmitter to a receiver; receiving a request for a different codedvideo stream and a switching stream from the receiver at thetransmitter; transmitting the requested switching stream from thetransmitter to the receiver; and transmitting the requested coded videostream from the transmitter to the receiver.
 6. The method of claim 5,further comprising: coding a plurality of video streams at a pluralityof bit rates at the transmitter; and storing each of the coded videostreams in a separate track at the transmitter.
 7. The method of claim6, further comprising transmitting from the transmitter to the receiverdata to inform the receiver of the plurality of bit rates correspondingto the stored coded video streams.
 8. The method of claim 5, furthercomprising storing each of a plurality of switching streams in aseparate track at the transmitter.
 9. The method of claim 5, furthercomprising storing in separate tracks at the transmitter at least threecoded video streams and at least four switching streams for each of aplurality of video streams.
 10. The method of claim 5, furthercomprising storing in separate tracks at the transmitter at least threecoded video streams and at least six switching streams for each of aplurality of video streams.
 11. A receiver-driven streaming system,comprising: a computer-processable medium; and logic stored on thecomputer-processable medium, the logic operable to receive an originalcoded video stream, to measure an available bit rate, to send a requestfor a different coded video stream and a switching stream based on theavailable bit rate, to receive the requested switching stream, and toreceive the requested coded video stream.
 12. A receiver-drivenstreaming system, comprising: a computer-processable medium; and logicstored on the computer-processable medium, the logic operable totransmit an original coded video stream, to receive a request for adifferent coded video stream and a switching stream, to transmit therequested switching stream, and to transmit the requested coded videostream.
 13. The system of claim 12, the logic further operable to code aplurality of video streams at a plurality of bit rates and to store eachof the coded video streams in a separate track and to transmit data toinform a receiver of the plurality of bit rates corresponding to thestored coded video streams.
 14. The system of claim 12, the logicfurther operable to store each of a plurality of switching streams in aseparate track.
 15. The system of claim 12, the logic further operableto store in separate tracks at least three coded video streams and atleast four switching streams for each of a plurality of video streams.16. The system of claim 12, the logic further operable to store inseparate tracks at least three coded video streams and at least sixswitching streams for each of a plurality of video streams.
 17. Areceiver-driven video stream, comprising: an original coded videostream; a different coded video stream and a switching stream requestedby a receiver measuring an available bit rate at the receiver andsending a request for the different coded video stream and the switchingstream based on the available bit rate.
 18. The receiver-driven videostream of claim 17, wherein the original coded video stream, thedifferent coded video stream and the switching stream are selected fromat least three coded video streams and at least four switching streams.19. The receiver-driven video stream of claim 17, wherein the originalcoded video stream, the different coded video stream and the switchingstream are selected from at least three coded video streams and at leastsix switching streams.
 20. The receiver-driven video stream of claim 17,wherein streaming of the original coded video stream, the differentcoded video stream and the switching stream is based at least in part onone or more hinting tracks each corresponding to one of a plurality ofcoded video streams coded at different bit rates and one or more hintingtracks each corresponding to one of a plurality of switching streams forswitching from a coded video stream at a first bit rate to a coded videostream at a second bit rate.